Skip to content

feat: unify tinfoil retry stack, add timeouts, surface real errors - #1151

Merged
ital0 merged 14 commits into
mainfrom
italomenezes/thu-747-tinfoilglm-reliability-unify-retry-stack-add-timeouts
Jul 30, 2026
Merged

feat: unify tinfoil retry stack, add timeouts, surface real errors#1151
ital0 merged 14 commits into
mainfrom
italomenezes/thu-747-tinfoilglm-reliability-unify-retry-stack-add-timeouts

Conversation

@ital0

@ital0 ital0 commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

GLM 5.2 via Tinfoil intermittently needed multiple retries; failures were invisible and the UI showed a generic error. This PR unifies the retry stack, bounds every hang vector, and surfaces the real failure cause.

What changed

Backend — Tinfoil proxy timeouts (backend/src/tinfoil/routes.ts)

  • Time-to-first-byte timeout (30s, configurable): upstream headers not arriving in time → 504 with a stable tinfoil upstream timeout marker the frontend classifies.
  • Stalled-stream watchdog (60s idle, configurable): mid-stream stalls abort the upstream and error the relayed stream (never truncate — bodies are opaque HPKE bytes, silent truncation would corrupt them). Implemented by extending the existing capStream (onIdle: 'terminate' | 'error', optional byte cap) instead of forking a second idle-relay implementation.
  • Client aborts now propagate to the enclave fetch (AbortSignal.any), so cancelled sends stop upstream work.

Frontend — SecureClient lifecycle (src/ai/tinfoil-client.ts, extracted from fetch.ts)

  • Attestation (ready()) is now bounded (15s): the SDK has no timeout/AbortSignal anywhere, and a chat send's abort signal does not cover the attestation await — a hung atc.tinfoil.sh previously hung the send forever. The bound applies to every call (not just the first) because an SDK-internal reset re-attests on the next ready().
  • A failed or timed-out ready() now evicts the cached client, so the next send reconstructs instead of inheriting a wedged instance.
  • Eviction predicate broadened: the SDK's concurrent-reset race escapes as a bare TypeError: … null (reading 'fetch') (no exported error class), which now also triggers eviction alongside KeyConfigMismatchError. All classification is by err.name — the tinfoil browser entry does not export the error classes, so instanceof is unusable.
  • The extraction realizes the code-split the prewarm path always claimed: prewarm-system-model, model-catalog, and the voice engine now import the light tinfoil-client module instead of dragging the whole AI-provider pipeline into the startup chunk.

Unified per-turn retry budget (src/ai/retry-budget.ts)

  • Previously three retry layers stacked multiplicatively with no shared cap: AI SDK internal (maxRetries default 2) × empty-response retry (×2) × outer chat auto-retry (×3).
  • Now one per-turn budget (6 requests / 120s wall-clock) lives on the chat instance and is consumed at the agent-agnostic routing choke point (works for the built-in pipeline, the Pi harness, and ACP agents alike). SDK-internal retries are disabled (maxRetries: 0) so every request is counted. A typed consumer/probe split prevents layers from double-drawing.
  • Budget resets on user send, manual retry, success, and abort — automation turns (sendAutomaticallyWhen) previously shared one budget until exhaustion.
  • Retry backoff is now actually exponential (2s/4s/8s + jitter); it was linear despite the comment.

Error UX + telemetry

  • Errors carry a kind through serialization (attestation / timeout / rate-limit / provider / network) and the chat error state shows cause-specific guidance instead of "Something went wrong" for every failure.
  • Minimal telemetry: tinfoil_attestation (outcome/duration/client, deduped to one success per client instance), chat_auto_retry gains a reason, plus chat_retry_success and chat_retries_exhausted.

Fault-injection coverage (all deterministic, DI-based, no module mocks)

ready() rejection/hang/timeout, construction failure, key-config mismatch, the null-transport race, re-attestation hang after internal reset, TTFB timeout, mid-stream stall, client abort propagation, connection reset cleanup, 429/4xx/5xx/timeout/network classification, budget exhaustion across layers, backoff progression, and budget reset semantics (including the automation path).

Verification

  • bun run check (type-check, lint, format, license): clean
  • bun run test: full frontend suite green
  • bun run test:backend: 938 pass, 0 fail
  • Tinfoil SDK behavior claims verified against tinfoil@1.1.11 dist (ready() semantics, internal retry counts, missing timeout/signal support, error names, EHBP framing)

Comment thread src/chats/chat-instance.test.ts Fixed
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

Semgrep Security Scan

No security issues found.

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

Preview environment destroyed 🧹

Stack preview-pr-1151 and its Cloudflare subdomain have been cleaned up.

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

PR Metrics

Metric Value
Lines changed (prod code) +698 / -212
JS bundle size (gzipped) 🟢 585.4 KB → 585.8 KB (+408 B, +0.1%)
Test coverage 🟢 81.00% → 81.12% (+0.1%)
Performance (preview) Preview not ready — Render deploy may have timed out
Accessibility
Best Practices
SEO

Updated Thu, 30 Jul 2026 19:49:48 GMT · run #2518

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔭 thunder-deep-review (advisory)

Reviewed the diff — no issues to report. ✅ Never approves, never requests changes, never gates merge.
head: 09c8f3202d51 · mode: deep · deferred 0 item(s) already reported by other bots (best-effort dedup)

@ital0 ital0 changed the title feat(THU-747): unify tinfoil retry stack, add timeouts, surface real errors feat: unify tinfoil retry stack, add timeouts, surface real errors Jul 29, 2026
@ital0 ital0 self-assigned this Jul 29, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔭 thunder-deep-review (advisory)

Complements the other bots — surfaces only what they did not flag. Never approves, never requests changes, never gates merge.
head: 44eb88983a0d · mode: deep · deferred 0 item(s) already reported by other bots (best-effort dedup)

Comment thread src/ai/fetch.ts

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔭 thunder-deep-review (advisory)

Complements the other bots — surfaces only what they did not flag. Never approves, never requests changes, never gates merge.
head: bf2cd1e4eaff · mode: deep · deferred 0 item(s) already reported by other bots (best-effort dedup)

Comment thread src/ai/fetch.ts Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔭 thunder-deep-review (advisory)

Complements the other bots — surfaces only what they did not flag. Never approves, never requests changes, never gates merge.
head: a7b3c96abc49 · mode: deep · deferred 0 item(s) already reported by other bots (best-effort dedup)

Comment thread src/ai/tinfoil-client.ts Outdated
Comment thread src/lib/error-utils.ts Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔭 thunder-deep-review (advisory)

Reviewed the diff — no issues to report. ✅ Never approves, never requests changes, never gates merge.
head: f321107bc0a7 · mode: deep · deferred 0 item(s) already reported by other bots (best-effort dedup)

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔭 thunder-deep-review (advisory)

Complements the other bots — surfaces only what they did not flag. Never approves, never requests changes, never gates merge.
head: 2afaa16f2091 · mode: deep · deferred 0 item(s) already reported by other bots (best-effort dedup)

Comment thread src/ai/retry-budget.ts
@ital0
ital0 merged commit 4dc166b into main Jul 30, 2026
31 checks passed
@ital0
ital0 deleted the italomenezes/thu-747-tinfoilglm-reliability-unify-retry-stack-add-timeouts branch July 30, 2026 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants